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IN THE CLAIMS 

Please cancel claims 37 and 51, and amend claims 15, 16, 30, 36, 43, 44, and 50 as 
follows: 

1-14 (Canceled) 

1 5 . (Currently Amended) A system for ind e xing raw data from at l e ast one data sourc e bas e d 
efi-detecting a semantic temporal event included in th e raw unprocessed video or audio data of an 
event from at least one data source, said system comprising: 

a knowledge-based modeling unit for generating multiple-layer models for identifying 
said semantic temporal event; 

a storage mechanism for storing said muhiple-layer models; 

an observation collection unit for extracting, from said [[raw]] unprocessed video or 
audio data from the at least one data source, temporal observations according to said multiple- 
layer models for the semantic temporal event : and 

a temporal event detection unit for detecting one or more occurrences of the semantic 
temporal event based on said temporal observations and said multiple-layer model s, wherein the 
semantic temporal event occurs during the event and the event has a start time and an end timet 

xxncc 

an indexing m e chanism for g e n e rating on ind e x for said raw data based on th e semantic 
temporal events det e cted by tho temporal e v e nt d e t e ction unit . 

16. (Currently Amended) The system according to claim 15, fiuther including: 

an a comput e riz e d event characterization unit for characterizing said occurrences of the semantic 
temporal event, detected by said temporal event detector, to produce a characterization for the 
occurrences of the semantic temporal event. 
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17. (Previously Presented) The system according to claim 16, further including: 
a storage mechanism for storing the characterization produced by said event 

characterization unit; 

an event prediction unit for performing temporal event prediction based on said 

characterization; 

an event model updating irnit for modifying said multiple-layer models based on said 
characterization; and 

an event simulation unit for simulating parts of said semantic temporal event according to 
said characterization. 

18-29 (Canceled). 

30. (Currently Amended) The system according to claim 15, wherein the semantic t e mporal 
event includes is_a sports event. 

3 1 . (Previously Presented) The system according to claim 30, wherein said sports event 
includes a soccer game. 

32. (Previously Presented) The system according to claim 15, wherein said multiple-layer 
models include a high level domain-specific knowledge model and a dynamic hierarchical event 
model. 

33. (Previously Presented) The system according to claim 32, wherein said high level 
domain-specific knowledge model includes rules of a sports game. 

34. (Previously Presented) The system according to claim 32, wherein said dynamic 
hierarchical event model includes a hierarchical decision tree. 
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35. (Previously Presented) The system according to claim 32, wherein said dynamic 
hierarchical event model includes an entity-relationship-diagram. 

36. (Currently Amended) The system according to claim 15, wherein said at least one data 
source includes at least on e data acquisition d e vice including is selected from the group 
consisting of a camera, a microwave sensor, a sound recorder, and an input data stream s e l e cted 
from the group consisting of vid e o, audio, t e xt, and temporal f e atur e s . 

37. (Canceled) 

38. (Previously Presented) The system according to claim 15, wherein said at least one data 
source includes a data stream sent through a network connection. 

39. (Previously Presented) The system according to claim 38 wherein the data stream is a 
video stream with synchronized audio track. 

40. (Previously Presented) The system according to claim 15, wherein the observation 
collection unit is simultaneously connected to more than one data source. 

41. (Previously Presented) The system according to claim 15, wherein said temporal event 
detection unit includes an integration unit, a detection unit, and a fusion unit, and said integration 
unit combines a plurality of observation streams from a plurality of data sources, the detection 
unit detects a same event using a plurality of detection means to produce a plurality of detection 
results, and said fusion unit fuses the plurality of detection results to produce a single detection 
decision. 

42. (Previously Presented) The system according to claim 16, further including: 

an event storage, in which detected occurrences of temporal semantic events are stored; 
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. an events statistics extractor to compute statistical information about the detected 
occurrences; and 

an event statistics storage unit to store the statistical information. 

43. (Currently Amended) A system for ind e xing raw data from at l e ast on e data sourco bas e d 
^ Mletecting a semantic temporal event included in tb»4 ^ unprocessed video or audio data of an 
event from at least one data source , said system comprising: 

a knowledge-based modeling imit for generating multiple-layer models for said semantic 
temporal event; 

a storage mechanism for storing said multiple-layer models; 

an observation collection unit for extracting, from said [[raw]] unprocessed video or 
audio data from said at least one data source, temporal observations according to said multiple- 
layer models for the semantic temporal event; 

a temporal event detection unit for detecting one or more occurrences of the semantic 
temporal event included in the [[raw]] unprocessed video or audio data based on said temporal 
observations and said multiple-layer models; 

an event characterization unit for characterizing said occurrences of the semantic 
temporal event, detected by said temporal event detector, to produce a characterization for the 
occurrences of the semantic temporal event; 

a storage mechanism for storing the characterization produced by said event 
characterization unit; 

an event prediction unit for performing temporal event prediction based on said 
characterization; 

an event model updating unit for modifying said multiple-layer models based on said 
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characterization; mid 

an event simulation unit for simulating parts of said semantic temporal event according to 
said characterizatio n, wherein the semantic temporal event occurs during the event and the event 
has a start time and an end time ^aitd 

an ind e xing mochanism for g e n e rating an ind e x for said raw data bas e d on at l e ast on e of fe e 
semantic temporal e v e nts d e t e cted by th e t e mporal e v e nt detection unit and th e charact e rizatie tt 
for th e occurr e nc e s of th e s e mantic temporal e v e nt . 

44. (Currently Amended) The system according to claim 43, wherein the s e mantic event 
includes is_a sports event. 

45. (Previously Presented) The system according to claim 44, wherein said sports event 
includes a soccer game. 

46. (Previously Presented) The system according to claim 43, wherem said multiple-layer 
models include a high level domain-specific knowledge model and a dynamic hierarchical event 
model. 

47. (Previously Presented) The system according to claim 46, wherein said high level 
domain-specific knowledge model includes rules of a sports game. 

48. (Previously Presented) The system according to claim 46, wherein said dynamic 
hierarchical event model includes a hierarchical decision tree. 

49. (Previously Presented) The system according to claim 46, wherein said dynamic 
hierarchical event model includes an entity-relationship-diagram. 

50. (Currently Amended) The system according to claim 43, wherein said at least one data 
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source includeG at loast one data acquiGition d e vioo including is selected from the group 
consisting of a camera, a microwave sensor, a sound recorder, and an input data stream s e l e cted 
from th e group consisting of video, audio, text, and t e mporal f e atur e s . 

51. (Canceled) 

52. (Previously Presented) The system according to claim 43, wherein said at least one data 
source includes a data stream sent through a network connection. 

53. (Previously Presented) The system according to claim 52, wherein the data stream is a 
video stream with synchronized audio track. 

54. (Previously Presented) The system according to claim 43, wherein the observation 
collection unit is simultaneously connected to more than one data source. 

55. (Previously Presented) The system according to claim 43, wherein said temporal event 
detection unit includes an integration unit, a detection unit, and a fusion unit, and said integration 
unit combines a plurality of observation streams from a plurality of data sources, the detection 
unit detects a same event using a plurality of detection means to produce a plurality of detection 
results, and said fiision unit fiises the plurahty of detection results to produce a single detection 
decision. 

56. (Previously Presented) The system according to claim 43, further comprising: 

an event storage, in which detected occurrences of temporal semantic events are stored; 

an events statistics extractor to compute statistical information about the detected occurrences; 
and 

an event statistics storage irnit to store the statistical information. 



600166357vl 



7 



